Touching is Believing: Sensing and Analyzing Touch Information with GelSight

For robots to perform advanced manipulation in a world of unknowns, touch is a critical source of information, and a high-quality tactile sensor is essential. However, existing tactile sensors generally are low-resolution and/or non-compliant, making it difficult to capture detailed contact information for manipulation that humans are very good at. GelSight was first developed a few years ago to capture micro-scale surface topography and converts pressure patterns to images, making it promising for highquality tactile sensing. However, the original devices were big, relatively slow, and expensive for robotic applications. In this work, we developed a new tactile sensor based on GelSight, which we call fingertip GelSight sensor, that is much more compact, faster and less expensive. Despite that, the fingertip sensor has high resolution, on the order of tens of microns, high compliance and high sensitivity. We demonstrated its unparalleled capabilities as a new-generation robotic fingertip for manipulation, in terms of localization and manipulation of small parts, normal and shear force estimation, and slip detection, as well as for material recognition, in terms of 3D surface texture classification. With image processing and machine learning techniques applied on the tactile images obtained, the fingertip GelSight sensor opens many possibilities for robotic manipulation that would otherwise be difficult to perform. Thesis Supervisor: Edward Adelson Title: Professor